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An Empirical Investigation of Four Tests for Interaction in the Context of Factorial 

Analysis of Covariance 

The Type I error and power properties of the parametric F test and three nonparametric 
competitors were compared in terms of a 3 x 4 factorial analysis of covariance layout. The 
focus of the study was on the test for interaction either in the presence and/or absence of 
main effects. A variety of conditional distributions, sample sizes, levels of variate and 
covariate correlation, and treatment effect sizes were investigated. The Puri and Sen 
(1969a) test had ultra-conservative Type I error rates and power losses when main 
effect(s) were present. The adjusted rank transform (Blair & Sawilowsky, 1990; Salter & 
Fawcett; 1993) had liberal Type I error rates when sampling was from moderate to 
extremely skewed distributions. The Hettmansperger (1984) chi-square test displayed 
acceptable Type I error rates for all distributions considered when sample sizes were ten 
or twenty. It is suggested that the Hettmansperger (1984) test be considered as an 
alternative to the parametric F test provided sample sizes are relatively equal and at least 
as large as ten. 
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An Empirical Investigation of Four Tests for Interaction in the Context of Factorial 

Analysis of Covariance 
Introduction 

The rank transform (RT) procedure was recommended as an alternative to the 
parametric procedure in multiple regression (Iman & Conover, 1979) and factorial 
analysis of covariance (Conover & Iman, 1981, 1982) when the assumption of population 
normality was violated. The steps for hypotheses testing using the RT consists of (a) 
replacing the raw scores with their respective rank order, (b) conducting the classical 
normal theory tests on the ranks, and (c) referring to the usual tables of percentage points. 

Unfortunately, the parametric F test is not invariant with respect to monotone 
transformations (such as the RT). More specifically, the nonlinear nature of the RT may 
add (remove) interactions when such interactions were absent (present) in the original 
raw scores. For example, and contrary to the suggestions above, it has been demonstrated 
that the RT fails as a viable alternative to the parametric procedure with respect to tests 
for (a) interaction in factorial ANOVA (Blair, Sawilowsky, & Higgins, 1987; Thompson, 
1991; 1993), (b) parallelism and interaction in factorial ANCOVA (Headrick, 1997; 
Headrick & Sawilowsky, 2000), and (c) additive and nonadditive models in multiple 
regression (Headrick & Rotou, 2000). 

However, nonparametric tests can be substantially more powerful than the 
parametric t or F tests when the assumption of normality is violated. For example, the 
Marm- Whitney U-test has an impressive asymptotic relative efficiency of 3 relative to the 
two independent samples t-test when the population sampled from is exponential 
(Conover, 1999). Thus, nonparametric or distribution free tests should be considered 
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when these tests demonstrate both (a) robustness with respect to Type I error and (b) a 
power advantage relative to the parametric test. 

Sawilowsky (1990) reviewed ten competing tests for interaction in the context of 
factorial ANOVA and ANCOVA. On the basis of Type I error and power properties, 
three potential competitors to the parametric F test remain. These alternative 
nonparametric tests are: the adjusted RT procedure (Blair & Sawilowsky, 1990; Salter & 
Fawcett, 1993); the Hettmansperger (1984) procedure; and the Puri and Sen (1969a) 
procedure. It should be noted that the Hettmansperger (1984) and Puri and Sen (1969a) 
procedures consider only the total group regression slope. As such, it is assumed that the 
within group regression slopes are equal for these tests. 

Purpose of the Study 

The purpose of the study is to compare and contrast the relative Type I error and 
power properties of the parametric F test and the three aforementioned nonparametric 
procedures in the context of factorial ANCOVA using Monte Carlo techniques. From the 
results of the Monte Carlo study, a statement will be made with respect to the conditions 
under which any of the nonparametric tests are useful alternatives to the parametric F 
test. Because good nonparametric tests exist for main effects, the focus of this study is 
concerned with the test for interaction in the presence and/or absence of main effects. 

Methodology 

A completely randomized balanced design with fixed effects and one covariate 
was used. The structural model representing the design was: 

Yyk=M + - ^) + > ( 1 ) 

( i = 1,..., /; j = 1 J\ and A:= 1,..., n), where /= 3, J= 4, and n = 5, 10, and 20. 
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The levels of variate ( ) and covariate ( ) correlation were /? = 0, .3, .6, and .9. Note 

that the regression slope coefficient in (1), /?, remained constant across groups. 

The treatment effect patterns modeled in (1) were as follows: 

1. The main effect r nonnull, the main effect a null, and the interaction (orr) null: 

1(a). r, =c?; 

1(b). T^ = T^ = d\ and =-d . 

2. The main effects r and a nonnull, and the interaction {ar) null: 

2(a). T^ = a^= d\ and r^= a^=-d\ and 

2(b). T^=a^=d\ and r^= t^= t^= a^= -d . 

3. The (orr) interaction nonnull, and the main effects rand a null: 

3(a). (orr),, = = c? ; and (arr),3 = = -d ; 

3(b). (arr)„ = (cirr),^ = = {az)^^ = c? ; and 

(arr),2 = (arr),3 = (arr)3, = (arr)34 = -d . 

4. The main effect r and the (orr) interaction nonnull, and the main effect a null: 

4(a). (err),, = d ; and (err), 4 = -d ; 

4(b). (err),, = (err),^ = (err)3, = {az)^^ = e? ; and 

(err),3 = (arr),4 = (arr)33 = (arr)34 = -d . 

5. The main effects r , er , and (err) interaction are nonnull: 

5(a). (err)2, = {az)^^ =d; 

5(b). (err),, = (err) ,3 = (err) 33 = (err) 33 = (err) 34 = e? ; and 

(err),3 = (err)3, = iaz)^, = -d . 
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The treatment effect sizes (d) ranged from J=0.10cr to d=2.00a, where cr is the 
standard deviation of the population from which samples were drawn, in increments of 

0.1 Oct The null case was represented when 0.00 for all effects. 

The tests for interaction for the competing procedures were conducted as follows: 

1 . The parametric F statistic was calculated using the OLS sums of squares 
approach given in Winer, Brown, and Michels (1991) for factorial ANCOVA. 
The F statistic for interaction was then compared to the critical value from the 
usual F tables of percentage points. 

2. The adjusted RT (adjRT) statistic was computed as follows: (a) the residuals 
were obtained from conducting a two-way ANOVA on the reduced model that 
included only the grouping variables; (b) the residuals and the covariate were 
then ranked without respect to group membership; and (c) the usual 
parametric ANCOVA procedure was conducted on the ranked residuals and 
ranked covariate to obtain the test statistic for interaction. This statistic was 
then compared to same critical F value as the parametric test. 

3. The Hettmansperger (H) (1984) chi-square statistic was computed as follows: 
(a) the residuals {RES) were obtained from the regression of the variate on the 
reduced model that included the covariate and the grouping variables; (b) the 
residuals were then ranked (denoted as RRES) without respect to group 
membership; (c) the standardized ranked residuals {SRRES) were obtained 



according to the following equation: SRRES = -s/ll 




SRRES were then submitted to a two-way ANOVA; (e) the sums of squares 



for interaction term obtained from the ANOVA was then compared to the 
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critical value from a chi-square distribution with (/ - 1)( J - 1) degrees of 
freedom (Hettmansperger, 1984). 

4. The Puri and Sen (PS) (1969a) chi-square statistic was computed as follows: 

(a) the variate and covariate were ranked irrespective to group membership; 

(b) the cell means ( Ryy , Rxy )> column means ( Ry j , Rx.j )> row means 

( Ryi , Ryj ), and overall grand means ( Ry , Rx „ ) were then obtained from the 

ranks of the variate and covariate scores; (c) the i/'-th difference score was 
then obtained as follows: 

DIFF{Ryy ) = {Ryy ~ Ry ) ~ {Ry J -RyJ~ (Ry, ~ Ry„ ) , aOd 

DIFF{R^y ) = (R^y - R^ ) - (Rx j - Rx .. ) - (Rxi. ~ ^x .. ) i (d) the ij-th residual 
scores were obtained from subtracting the predicted differences from the 
observed differences as follows: RESy = DIFF{Ryy^ ) - Py^DIFF(Rj^y ) , 

where Py^ is the total group rank correlation coefficient between the variate 
and covariate; (e) the statistic (Puri & Sen, 1969a) was then formulated as: 

Rn - ^11 ^11 the first element on the principal 

i J 

diagonal of the inverted variance-covariance matrix ( V ); and (f) the 
computed value of was subsequently compared to the critical value from a 
chi-square distribution with (/ - 1)( J - 1) degrees of freedom (Puri & Sen, 
1969a). 

Nine conditional distributions were simulated with zero means ( // = 0 ), unit 
variances ( = 1 ), and varying degrees of /i , /2 > /a > . The distributions 
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approximated in the simulation were: 1 =normal (/, =0, =0, /j =0, and y^ =0), 2 = 

uniform (/, =0, “ “6/5 , /j =0, and y^ = 48/7); 3 = Cauchy (/, =0, y^=15 , 

y^ =0, and y^ = 4000 ); 4= double exponential (/i =0, = 3 , /j =0, and y^ = 30); 

5 = logistic ( /, = 0, ^2 = 6/5 , /3 = 0, and ^4 = 48/7 ); 6 = chi-square Mf{y^=\, 
y^ = 2>ll , ^3 =3, and y^ = 15/2 ), 7 =chi-square 4^/f(/, =-j2 , y^ =4, y^ = bV2 , and 
y^ =30), 8= chi-square 2df (/, =2, y^ =6, y^ =24, and y^ = 120), and 9= chi-square 1 
y^=\2, y^= 48 V 2 , and y^ =480). The preceding values of (coefficient 
of skew), ^2 (coefficient of kurtosis), y ^ , and y^ are the third, fourth, fifth, and sixth 

standardized cumulants from their associated probability density functions with the 
exception of the Cauchy distribution. Because the moments of a Cauchy pdf are infinite, 
the above values of y^, y 2 , y^, and y^ associated with this density were selected to 

yield a symmetric distribution with heavy tail-weight. 

The steps employed for data generation follow the model developed by Headrick 
(2000). The Headrick (2000) procedure is an extension of the Headrick and Sawilowsky 
(1999, 2000) procedure for simulating multivariate nonnormal distributions. The 
Headrick (2000) procedure generated the Y..^^ and for the i>th group in (1) from the 



use of the following equations: 



'^ijk “ Cq + c, + C 2 '^^i^ijk + ^ijd and 

iik ~ Cq '^^S-^ijk' 



( 2 ) 

( 3 ) 



The resulting and X.^ were distributed with group means of Syd and zero 



(respectively), unit variances, the desired values of /i , /2 > T'a > A > the desired 
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within group correlation (/?). In all experimental situations, Yyj^ and followed the 
same distribution. The value of Syd was the shift parameter added to the i/'-th group for 
the treatment effect pattern considered. The coefficients c^, c, , c^, Cj , c^, and Cj were 
determined by simultaneously solving equations 37, 38, 39, 40, 41, and 42 from Headrick 
(2000) for the desired values of , /2 > /s > A ■ values of and in (2) 
and (3) were generated using the following algorithms: 



where the Zy ^ , Vy ^ , and Wy,^ ~ N iid(0,l). The resulting Yy^ and Xy^ were normally 
distributed with zero means, unit variances, and correlated at the intermediate value 
p’\ . The intermediate correlation, which is different from the desired post-correlation 



iPv X ) except under conditional normality, was determined by solving equation 26 

from Headrick (2000) for the bivariate case for /?*.^. ■ When both variables follow the 
same distribution, equation 26 from Headrick (2000) can be expressed as follows: 



Values of Cg ,. . ., c, , and /?* . were solved for (6) using Mathematica (Version 4.0, 




( 4 ) 




( 5 ) 




1999). The solution values of c„ ,. . ., c, , the intermediate correlations ( p*. ), and post- 

correlations ( ) for the conditional distributions considered are compiled in Table 1. 



The computer used to carry out the Monte Carlo was a Pentium Ill-based personal 
computer. All programming was done using Lahey Fortran 77 version 3.0 (1994), 
supplemented with various subroutines from RANGEN (Blair 1986). Using the chi- 
square and F tables of percentage points, the proportions of hypotheses rejected were 
recorded for the four different procedures. The nominal alpha level selected was .05. 
Twenty five thousand repetitions were simulated for each of the 9(type of 
distribution) x4(level of correlation)x21(effect size) x 10(treatment effect pattern) 
experiments. 

Results 

Adequacy of the Monte Carlo 

For each repetition, separate values of Py and , and for the 

variate and covariate for each of the //groups were computed. Average values of 
PijiP. )> ri, (/■..)> (r2..)> ri, iK. )> and y,^ {y,_) were obtained by averaging the 

Pu, 7x , Yt , , and y, across the //groups. The values of jo , , /2 > > and 

y^ were subsequently averaged across 25,000 (replications) x 21 (effect size) situations 

in the first treatment effect pattern for each conditional distribution. The average values 
of /i , / 2 .. > ^ 3 .. > and y^ were then further averaged across the four levels of correlation. 

The overall averages of , /2 > /3 > /4 > and p are listed in Table 2 and Table 3, 
respectively. Inspection of Tables 2 and 3 indicate that the Headrick (2000) procedure 
produced excellent agreement between /i , /2 > ^3 > A > and p and the population 
parameters considered. 
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Table 1. Values of constants ( ) used in equation (3), population correlations 
{ Pv V ), and intermediate correlations (/?*! ) to simulate and correlate the desired 

conditional distributions (Dist). 



Dist . 


Co 


Cl 


^2 


C3 


C4 


C5 




Py * x * 


1 


0.000000 


1.000000 


0.000000 


0.000000 


0.000000 


0.000000 


.00 


.000000 
















.30 


.300000 
















.60 


.700000 
















.90 


.900000 


2 


0.000000 


1.347438 


0.000000 


- 0.140177 


0.000000 


0.001808 


.00 


.000000 
















.30 


.326197 
















.60 


.634118 
















.90 


.913613 


3 


0.000000 


0.306093 


0.000000 


0.184686 


0.000000 


0.001132 


.00 


.000000 
















.30 


.374236 
















.60 


.683980 
















.90 


.929263 


4 


0.000000 


0.727709 


0.000000 


0.096303 


0.000000 


- 0.002232 


.00 


.000000 
















.30 


.309371 
















.60 


.612882 
















.90 


.905531 


5 


0.000000 


0.879467 


0.000000 


0.040845 


0.000000 


- 0.000405 


.00 


.000000 
















.30 


.302233 
















.60 


.603260 
















.90 


.901368 


6 


- 0.163968 


0.950794 


0.165391 


0.007345 


- 0.000474 


0.000014 


.00 


.000000 
















.30 


.311431 
















.60 


.612677 
















.90 


.904625 


7 


- 0.227508 


0.900716 


0.231610 


0.015466 


- 0.001367 


0.000055 


.00 


.000000 
















.30 


.322263 
















.60 


.624030 
















.90 


.908552 


8 


- 0.307740 


0.800560 


0.318764 


0.033500 


- 0.003675 


0.000159 


.00 


.000000 
















.30 


.341958 
















.60 


.643339 
















.90 


.914879 


9 


- 0.397725 


0.621071 


0.416907 


0.068431 


- 0.006394 


0.000044 


.00 


.000000 
















.30 


.376853 
















.60 


.673908 
















.90 


.924127 
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T able 2. Average values of ( /, ), ( fi )> ^3 ( /a )> A ( / 4 ) simulated by the 

Headrick (2000) procedure. The average values (/, , /a > ^ 4 ) listed below were based 
on a sample size is « = 20. 



Distribution 




Population parameter ( , /2 > /3 > /4 ) 




1 

Variate (Y) 
Covariate (X) 


r\ =0 

=0.000124 
y =-0.000084 


/2 =0 

f2 =-0.000284 
^2 =0.000452 


Xi =0 

/3 =0.001073 
/3 =0.000795 


74 =0 

=-0.001339 

=0.002845 


2 

Variate (Y) 
Covariate {X) 


r\ =0 

=0.000005 
fx =0.000039 


Xi = -6/5 
f2 =-1.200004 
f2 =-1.200163 


Xi =0 

/3 =0.0000238 
/3 =0.0001685 


74 = 48/7 
/4 =6.857894 
/4 =6.853492 


3 

Variate (Y) 
Covariate (X) 


r\ =0 

=-0.001318 
fx =0.000290 


^2=25 

/2 =24.975520 
/2 =24.941770 


Xi =0 

/3 =-.3386690 
/3 =-0.799517 


/4 =4000 
/4 =3958.221140 
/4 =3988.304000 


4 

Variate (Y) 
Covariate (X) 


r\ =0 

fx =0.000342 
fx =0.000032 


Xi =3 

f2 =2.999848 
f2 =3.000327 


/3 =0 

=0.014447 
/3 =0.004328 


74=30 

/4 =30.010830 
/4 =30.006732 


5 

Variate {Y) 
Covariate {X) 


r\ =0 

=0.000224 
fx =0.000034 


Xi = 6/5 
=1.199900 
f2 =1.200087 


/3 =0 

/3 = .004258 
/3 = .001478 


74 = 48/7 
/4 =6.846827 
/4 =6.858595 


6 

Variate {Y) 
Covariate {X) 


rx =1 

fx =1.000071 
fx =0.999992 


X2 = 3/2 
= 1.500197 
/2 =1.500053 


/3 =3 

/3 =3.001597 
/3 =3.005218 


74 = 15/2 
/4 =7.496629 
/4 =7.538564 


1 

Variate (Y) 
Covariate (X) 


ri = V 2 
fx =1.414330 
fx =1.413904 


Xi =3 

/2 =3.000764 
/2 =3.001067 


/3 = 6 V 2 

/3 = 8.489000 
=8.484897 


74 =30 

/4 =29.978800 
/4 =30.004765 


8 

Variate (y) 
Covariate (X) 


rx =2 

fx =2.000254 
fx =1.999989 


Xi =6 

/2 =6.002129 
f2 =6.000573 


73=24 

/3 =24.008980 
/3 =24.010045 


74 = 120 

/4 = 119.868700 

/4 =120.158647 


9 

Variate (y) 
Covariate {X) 


/i = Vs 

fx =2.828878 
fx =2.827901 


/2 =12 

/2 = 12.003800 
f2 =12.000050 


/3 = 48 V 2 

/3 =67.884840 
/3 =67.885672 


/4 =480 
/4 =479.035600 
/4 =480.001874 
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Table 3. Average values of variate and covariate correlation (p) simulated by the 
Headrick (2000) procedure. The value p denotes the population correlation. The average 



values (p) listed below were based on a sample size is n=20. 













Distribution 








1 


2 


3 


4 


5 


6 


7 


8 


9 


n 


p 


p 


p 


p 


p 


p 


P 


p 


P 


p 


20 


.00 


.000 


.000 


-.001 


.000 


.000 


.000 


.001 


.000 


-.000 




.30 


.300 


.299 


.300 


.301 


.300 


.299 


.300 


.300 


.301 




.60 


.600 


.601 


.602 


.599 


.600 


.598 


.600 


.599 


.600 




.90 


.900 


.899 


.901 


.900 


.900 


.901 


.899 


.900 


.900 



The Type I error and power analyses are compiled in Tables 4 through 13. The 
colunm entries from left to right denote (a) the test statistic, (b) the standardized 
treatment effect size “d", and (c) the proportion of rejections for the four different tests of 
interaction under the various levels of variate and covariate correlation and the other 
parameters considered. 

Type I Error 

Normal Distribution: The Type I error rates for the competing procedures are 
compiled in Tables 4, 6, and 8, for n = 5, 10, 20, and treatment pattern 2(b). This 
particular effect pattern is reported because the commonly used rank transform test 
statistic (Conover & Iman, 1981) under these circumstances is not asymptotically chi- 
squared (Thompson, 1991, 1993) and is liberal for even small samples (Headrick, 1997; 
Headrick & Sawilowsky, 2000). 

As expected, the parametric F test maintained Type I error rates close to nominal 
alpha and were within the closed interval of a±l.96^a(\ - a) 125000 . This occurred 

across all treatment conditions, sample sizes, and levels of variate/covariate correlation. 

The adJRT also generated acceptable Type I error rates. Inspection of Tables 4, 6, 
and 8 indicates that the Type I error rates were similar to the parametric F test. 
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With respect to the H test, inspection of Tables 6 and 8 indicates that this test 
maintained appropriate Type I error rates for sample sizes of n = 1 0 and n = 20. However, 
for n = 5, inspection of Table 4 indicates that the H test generated liberal Type I error 
rates. For example, with an effect size of 0.80, the Type I error rates were 
approximately .060 across all levels of variate/covariate correlation. 

The PS test became conservative when either one or both main effects were 
present. Ceteris paribus, the stronger the noimull main effect(s) the more conservative the 
Type I error rates became. These conservative Type I error rates occurred across all 
levels of variate and covariate correlation. For example, with an effect size of 0.80, 
inspection of Table 4 indicates that the Type I error rates were .001, .000, and .000 across 
the three levels of variate/covariate correlation. The PS procedure maintained Type I 
error rates close to nominal alpha only when both main effects were null. 

Nonnormal Distributions'. Type I error rates are compiled in Tables 10 and 12 for 
some of the noimormal distributions considered. The approximate distributions reported 
in these tables are the chi-square \df and Cauchy. These distributions are reported 
because previous empirical investigations demonstrated that Type I error inflations 
associated with the rank transform test statistic (Conover & Iman, 1981) were most 
severe under extreme departures from normality (Headrick, 1 997 ; Headrick & 
Sawilowsky, 2000). 

The parametric F test was slightly conservative under the noimormal conditional 
distributions reported. For example, with an effect size of <i= 1 .30, variate/covariate 
correlation of r = .30, an inspection of Table 12 indicates that the Type I error rate was 
.040 when the conditional distribution was approximate Cauchy. 
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The adjRT generated inflated Type I error rates when the conditional distribution 
considered was skewed (e.g., chi-square \df or 2df). For example, with an effect size of 
^/=0.80, a variate/covariate correlation of r=.90, inspection of Table 10 indicates that the 
Type 1 error rate for the adjRT was .076. In general, increases in skew i.e., chi-square 
4df, chi-square 2df, chi-square l^^were associated with increases in Type I error inflation 
for the adjRT. 

The H test maintained appropriate Type I error rates for all nonnormal conditional 
distributions considered when sample sizes were n = 10 and n =20. When samples were 
n = 5, the H test generated liberal Type I error rates. The inflated Type I error rates were 
similar to those error rates generated under conditional normality. 

As with the standard normal case, the PS test generated ultra-conservative Type I 
error rates when main effects were present. For example, with an effect size ^/=0.80 and 
a variate/covariate correlation of r= .60, inspection of Table 12 indicates that the Type I 
error rate was .000. This occurred for all nonnormal distributions considered in this 
study. 

Power Analysis 

Normal Distribution'. Power analyses for the competing procedures are compiled 
in Tables 5, 7, and 9, for n = 5, 10, 20, and treatment pattern 5(a). This effect pattern is 
reported because under these conditions the usual rank transform statistic has been 
demonstrated to display severe power losses (Headrick, 1997; Headrick & Sawilowsky, 
2000 ). 

As expected, the F test displayed a power advantage over the three nonparametric 
competitors when the conditional distribution was standard normal. Specifically, the F 
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test was substantially more powerful than the PS test when both main effects became 
increasingly nonnull. Although the F test was more powerful than the H test when sample 
sizes were « = 10 and n = 20, the H test held a slight power advantage over the adjRT. 
When sample sizes were n = 5, inspection of Table 5 indicates that the H test was 
rej ecting at a higher rate than F test. For example, with an effect size of 0.80, a 
variate/covariate correlation of r= .30, inspection of Table 5 indicates that the H test had 
a rejection rate of .16 while the F test was rejecting at a rate of .145. This higher rejection 
rate is attributed to the liberal nature of the Type I error rates that were associated with 
the H test when « = 5. 

Nonnormal Distributions'. In general, when departures from normality were small 
(e.g., approximate logistic) to moderate (e.g., approximate chi-square Mf) the F test 
rejected at rates slightly less than the Hettmansperger and adjRT procedures. The power 
advantages in favor of either the H or adjRT tests were contingent on the conditional 
distribution considered and the other parameters being simulated. It should be noted that 
the power advantages in favor either the H test or adjRT test were marginal. On the other 
hand, when the conditional distribution was approximate uniform the parametric F test 
held a slight advantage over the nonparametric procedures. 

When the conditional distributions were extremely skewed and/or heavy tailed, 
both the adjRT and H tests held large power advantages over the F test. Further, when the 
adjRT test generated reasonable Type I error rates, the adjRT displayed some power 
advantages over the other competing nonparametric procedures. For example, inspection 
of Table 1 3 indicates that when the conditional distribution was approximate Cauchy, an 
effect size of 0.80, and a variate/covariate correlation of r= .60, the adjRT was 
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rejecting at a rate of .942 whereas the H test was rejecting at a rate of .844. Power 
comparisons between these two tests were not considered where the adjRT generated 
liberal Type error rates (e.g., approximate chi-square Idf or chi-square \df). 

When the conditional distributions were approximately chi-square 2df or chi- 
square \df, the H test was a much more powerful than the parametric F. For example, 
when sampling was from an approximate chi-square distribution \df, c?=0.80, a 
variate/covariate correlation of r= .30, inspection of Table 1 1 indicates that the H test was 
rejecting at a rate of .731 while the F test was rejecting at a rate of only .326. 

The PS procedure held a power advantage over the H and adjRT tests only when 
both main effects were either weak or null. Otherwise, the PS test statistic had the 
problem of power loss when juxtaposed to either the H or the adjRT tests as the 
magnitude of the main effect(s) increased. For example, when sampling was from an 
approximate chi-square distribution \df, c?=0.30, a variate/covariate correlation of r=.30, 
inspection of Table 1 1 indicates that the PS test was rejecting at a rate of .182 while the H 
test was rejecting at a rate of . 148. However, when the effect size increased from d= 30 
to <i=0.80, the H test was rejecting at a rate of .731 while the PS was rejecting at a rate of 
only .524. This pattern of power loss associated with the PS test was consistent across all 
nonnormal distributions considered in this study. 

Discussion 

The PS test is computationally arduous. Further, the results of this study indicate 
that this test had the problems of ultra-conservative Type I error rates and power loss 
when main effects were nonnull. Toothaker and Newman (1994) found similar results 
with respect to the PS test in the context of factorial ANOVA. Thus, it is recommended 
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that this procedure not be considered as a viable alternative to the parametric F test in 
factorial ANCOVA. 

It is possible to base the PS statistic on normal or expected normal scores instead 
of the ranks (Puri & Sen, 1969a). And, this might correct the problem of ultra- 
conservative Type I error rates. However, additional nonlinear transformations present 
the problem with respect to the correct interpretation of the statistical results in terms of 
the original metric. 

The adjRT is arguably the simplest of the three nonparametric procedures to 
compute. However, because the adjRT has the problem of liberal Type I error rates when 
the distributions possess moderate to extreme skewness, it is also recommended that the 
adjRT procedure not be used in place of the parametric Ftest. 

The H chi-square test maintained appropriate Type I error rates for all conditional 
distributions considered in this study when sample sizes were at least as large as « = 10. 
Thus, the H test could be considered as an alternative to the parametric F test for 
interaction provided the within group sample sizes are relatively equal and at least as 
large as « = 10. This recommendation is made in view of the large power advantage that 
the H test had over the F test when the conditional distributions were contaminated with 
outliers and/or possessed extreme skewness. 
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Table 4. Type I error results for the test of interaction. The sampling distribution was 
standard normal. The sample size was n=5. Both main effects were nonnull. The Type I 
error rates were based on 25,000 repetitions and a nominal alpha level of or = .05 . 


Test 




Level of Correlation 


Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.051 


.052 


.050 


adJRT 




.052 


.053 


.051 


H 




.057 


.060 


.060 


PS 




.023 


.020 


.004 


F 


0.80 


.050 


.051 


.050 


adJRT 




.052 


.053 


.049 


H 




.059 


.056 


.058 


PS 




.001 


.000 


.000 


F 


1.30 


.052 


.047 


.052 


adJRT 




.052 


.050 


.051 


H 




.059 


.060 


.061 


PS 




.000 


.000 


.000 



Table 5. Power analysis for the test of interaction when sampling was from a standard 
normal distribution. The sample size was n=5. Both main effects were nonnull. The 
rejection rates were based on 25,000 repetitions and a nominal alpha level of or = .05 . 



Test 






Level of Correlation 




Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.062 


.067 


.110 


adJRT 




.062 


.066 


.100 


H 




.069 


.077 


.121 


PS 




.055 


.056 


.081 


F 


0.80 


.145 


.202 


.622 


adJRT 




.143 


.187 


.531 


H 




.159 


.217 


.632 


PS 




.106 


.132 


.315 


F 


1.30 


.359 


.507 


.983 


adJRT 




.349 


.473 


.954 


H 




.'ill 


.517 


.983 


PS 




.211 


.272 


.272 
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Table 6. Type I error results for the test of interaction. The sampling distribution was 
standard normal. The sample size was n= 10. Both main effects were nonnull. The Type I 
error rates were based on 25,000 repetitions and a nominal alpha level of a = .05 . 



Test 




Level of Correlation 




Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.053 


.052 


.049 


adjRT 




.054 


.051 


.049 


H 




.052 


.055 


.053 


PS 




mi 


.019 


.003 


F 


0.80 


.050 


.050 


.050 


adjRT 




.051 


.049 


.051 


H 




.053 


.052 


.053 


PS 




.006 


.001 


.000 


F 


1.30 


.050 


.048 


.050 


adjRT 




.051 


.048 


.051 


H 




.054 


.051 


.054 


PS 




.000 


.000 


.000 



Table 7. Power analysis for the test of interaction when sampling was from a standard 
normal distribution. The sample size was n = 10. Both main effects were nonnull. The 
rejection rates were based on 25,000 repetitions and a nominal alpha level of a = .05 . 



Test 




Level of Correlation 


Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.078 


.088 


.208 


adjRT 




mi 


.087 


.179 


H 




.076 


.089 


.201 


PS 




.069 


mi 


.155 


F 


0.80 


.294 


.418 


.951 


adjRT 




.284 


.386 


.911 


H 




.288 


.402 


.943 


PS 




.232 


.306 


.779 


F 


1.30 


.715 


.879 


1.000 


adjRT 




.693 


.848 


.999 


H 




.697 


.863 


1.000 


PS 




.531 


.683 


.987 
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Table 8. Type I error results for the test of interaction. The sampling distribution was 
standard normal. The sample size was n=20. Both main effects were nonnull. The Type I 
error rates were based on 25,000 repetitions and a nominal alpha level of or = .05 . 



Test 


w/ ---- — 




Level of Correlation 




Effect Size (<i) 


0.3 


0.6 


0.9 


F 


0.30 


.050 


.050 


.049 


adjRT 




.050 


.050 


.050 


H 




.051 


.052 


.050 


PS 




.028 


.019 


.003 


F 


0.80 


.051 


.049 


.052 


adjRT 




.052 


.052 


.051 


H 




.052 


.052 


.052 


PS 




.001 


.000 


.000 


F 


1.30 


.050 


.050 


.050 


adjRT 




.050 


.049 


.048 


H 




.052 


.051 


.052 


PS 




.000 


.000 


.000 



Table 9. Power analysis for the test of interaction when sampling was from a standard 



normal distribution. The sample size was n=20. Both main effects were nonnull. The 
rejection rates were based on 25,000 repetitions and a nominal alpha level of a = .05 . 


Test 




Level of Correlation 


Effect Size (d) 


0.3 


0.6 


0.9 


F 


0.30 


.109 


.133 


.410 


adjRT 




.105 


.127 


.360 


H 




.105 


.131 


.393 


PS 




.099 


.119 


.328 


F 


0.80 


.596 


.775 


1.000 


adjRT 




.569 


.734 


1.000 


H 




.573 


.754 


1.000 


PS 




.505 


.661 


.994 


F 


1.30 


.976 


.998 


1.000 


adjRT 




.968 


.996 


1.000 


H 




.969 


.997 


1.000 


PS 




.920 


.978 


1.000 
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Table 10 . Type I error results for the test of interaction. The sampling distribution was an 
approximate chi-square distribution with 1 degree of freedom. The sample size was n = 10. 
Both main effects were nonnull. The Type I error rates were based on 25,000 repetitions 



and a nominal alpha level of a = .05 . 



Test 




Level of Correlation 


Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.046 


.043 


.045 


adjRT 




.069 


.068 


.075 


H 




.053 


.048 


.048 


PS 




.010 


.006 


.001 


F 


0.80 


.044 


.047 


.047 


adjRT 




.067 


.072 


.076 


H 




.051 


.050 


.049 


PS 




.004 


.001 


.000 


F 


1.30 


.045 


.046 


.046 


adjRT 




.070 


.070 


.073 


H 




.052 


.049 


.049 


PS 




.000 


.000 


.000 



Table 11. Power analysis for the test of interaction when sampling was from was an 
approximate chi-square distribution with 1 degree of freedom. The sample size was n = 10. 
Both main effects were nonnull. The rejection rates were based on 25,000 repetitions and 
a nominal alpha level of a = .05 . 


Test 




Level of Correlation 


Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.075 


.086 


.221 


adjRT 




.197 


.262 


.679 


H 




.148 


.182 


.522 


PS 




.182 


.243 


.597 


F 


0.80 


.326 


.462 


.947 


adjRT 




.815 


.926 


.999 


H 




.731 


.838 


.999 


PS 




.524 


.672 


.973 


F 


1.30 


.739 


.881 


.999 


adjRT 




.991 


.999 


1.000 


H 




.981 


.995 


1.000 


PS 




.762 


.885 


.998 
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Table 12. Type I error results for the test of interaction. The sampling distribution was an 
approximate Cauchy distribution. The sample size was n = 10. Both main effects were 
nonnull. The Type I error rates were based on 25,000 repetitions and a nominal alpha 
level of a = .05. 



Test 




Level of Correlation 


Effect Size (^f) 


0.3 


0.6 


0.9 


F 


0.30 


.043 


.045 


.044 


adJRT 




.054 


.053 


.058 


H 




.046 


.050 


.050 


PS 




.008 


.005 


.001 


F 


0.80 


.044 


.044 


.046 


adJRT 




.053 


.055 


.056 


H 




.045 


.048 


.048 


PS 




.000 


.000 


.000 


F 


1.30 


.040 


.044 


.045 


adJRT 




.053 


.052 


.056 


H 




.045 


.047 


.048 


PS 




.000 


.000 


.000 



Table 13. Power analysis for the test of interaction when sampling was from was an 
approximate Cauchy distribution. The sample size was n = 10. Both main effects were 
nonnull. The rejection rates were based on 25,000 repetitions and a nominal alpha level 



of a = .05. 



Test 




Level of Correlation 


Effect Size {d) 


0.3 


0.6 


0.9 


F 


0.30 


.075 


.096 


.244 


adJRT 




.163 


.235 


.699 


H 




.130 


.173 


.495 


PS 




.155 


.220 


.632 


F 


0.80 


.346 


.489 


.946 


adjRT 




.801 


.942 


1.00 


H 




.712 


.844 


.999 


PS 




.563 


.754 


.993 


F 


1.30 


.750 


.884 


.999 


adjRT 




.993 


.999 


1.00 


H 




.981 


.996 


1.00 


PS 




.804 


.934 


.999 
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